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(57) Abstract 

Raw digital video data is generated by a digital video camera and transmitted over a bus (27) to a computer (28). The raw digital 
video data is generated from the analog output of a charge coupled device (23) in the camera, which preferably includes color filter overlays. 
The bus can be a relatively low-speed bus, having a bandwidth of not greater than 12 Mbils/sccond. The video camera preferably processes 
the raw digital video data prior to transmitting it over the bus to decrease the resolution of the image defined by the raw digital video 
data. Such processing can be downsampiing in both the horizontal and vertical dimensions. The computer includes a processor capable of 
converting the raw digital video data into usable digital video data, such as YUV or YIQ digital video data. 
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COLOUR VIDEO DATA REDUCTION 
BACKGROUND OF THE INVENTION 

J Field of the Invention 

This invention relates to the field of video color format and compression. 

2. Description of the Related Art 

The advent of powerful microprocessors has expanded the quantity and quality of 
computer applications. Multimedia and other video related applications in particular are 
increasingly prominent. These applications typically involve transmitting video data into and out 
of personal computers (PCS). In typical video related applications, as shown in Fig. 1, a digital 
video camera 10 is used to capture video 18. Digital video camera 10 typically includes a lens 
12, a single plate charge coupled device (CCD) sensor 13, correlated double sampling and 
automatic gain control device 14, analog to digital (A/D) converter 15, digital signal processor 
(DSP) 1 6 and microcontroller 1 7. 

As noted, typical video cameras for PC and home applications use a single plate CCD. 
Therefore, to produce a color video signal, the CCD's are covered with a thin film color filter, 
which can have various color formats. For example, there are filters having primary colors: red, 
green and blue (RGB); and filters with complimentary colors: cyan, yellow, green, magenta 
(Cy,Y,G,Mg) and white, cyan, yellow and green (W,Cy,Y,G). 

The correlated double sampling and automatic gain control device 14 conditions the 
analog signal output by the CCD 1 3 and passes the conditioned signal to A/D converter 1 5. The 
analog video signal is converted to a digital video signal by analog/digital converter 15 and the 
digital signal is passed to DSP 16. DSP 16 performs color transformation to, for example, the 
YUV, which is the predominant choice both in the digital television industry and for PC digital 
video and teleconferencing applications, or YIQ format and further manipulates image quality, as 
is known to those having ordinary skill in the art. In fact, traditional video cameras perform a 
large amount of analog and digital signal processing. The hardware that implements this 
processing is responsible for a considerable amount of camera cost. 

Microcontroller 17 controls the operation of DSP 16 as understood by those having 
ordinary skill in the art. 
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The YUV format nominally includes a total of 24 bits of data, 8 bits for each of the Y, U 
and V components. Typically, the YUV signal is subsampled, for example by DSP 16 prior to 
transmission to PC 1 1 to reduce the number of bits which must be transmitted through bus 19. 
Popular subsampled YUV formats include YUV 16, YUV 12 and YUV9. 

In YUV9, for example, pixels are grouped into 4x4 blocks, for a total of 16 pixels. Each 
of the 16 pixels maintains its separate 8 bit Y luminance information. Each pixel, however, 
shares a single 8 bit U and a single 8 bit V chrominance component. The tola! number of bits 
which must be sent to represent each pixel is 9: ((16 Y * 8 bits) + (lU * 8 bits) + (I V * 8 
bits))/ 16 pixels - 144 bits/ 16 pixels = 9 bits per pixel (9bpp). YUV9 subsampling thereby 
reduces the data required to represent one pixel from 24 bits to 9 bits without a severe effect on 
image quality, since human viewers have poor acuity for color transitions in an image but 
relatively high acuity to intensity level changes. 

YUV 16 and YUV 12 color subsampling employ similar techniques on a smaller matrix of 

pixels. 

After converting digitized raw CCD color information to, for example, a YUV format and 
subsampling the data, the camera must transmit the video data to PC 11, for example for storage 
or display on an attached monitor (not shown in Fig. I). The video data typically is transmitted 
to PC 1 1 through a parallel or serial port and over Universal Serial Bus (USB) 19. 
Unfortunately, even the most advanced USB has a theoretical bandwidth limit of 12Mbit/second, 
the maximum sustainable usable rate is more accurately about 6 Mbit/second. The bandwidth 
limitations imposed by the USB have a significant adverse effect on the quality of ftill motion PC 
video. For example, the following table shows maximum sustainable frame rates at various 
resolutions for YUy9 and YUV 12 data through a USB: 



Color\ 
Resolution 


YUV9 


YUV12 


160 * 120 


34.7 fps 


26.0 fps 


320 * 240 


8.7 fps 


6.5 fps 



Table 1, Video Transmission Frame Rates for YUV Color Formats 

The minimum refresh rate for acceptable full motion video is at least 30 fps. As can be 
seen from the above table, a USB bus is capable of delivering only YUV9 subsampled video data 
at low resolution at the rate required for acceptable full-motion video. 
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A system and method are needed, therefore, for transmitting digital video signals from 
digital video camera at a rate sufficient to reproduce ftill motion video over transmission media 
having bandwidth constraints. 



a 



^ SUMMARY OF THF rNVENTION 

An image is captured by a camera which has a sensor for generating a plurality of analog 
video signals in response to the image. The analog video signals are converted into a plurality of 
raw digital video signals defining the image. The raw digital video signals are transmitted to a 
computer over a bus. In an embodiment, the resolution of the image defined by the raw digital 
1 0 video signals is decreased prior to being transmitted over the bus. 

BRIEF DRSCRTPTin N OF THR PR AWINH^ 
The following detailed description will be more fully understood with reference to the 
accompanying drawings in which: 
' ^ ^^B- 1 is a block diagram of a prior art digital camera to PC via bus coupling; 

Fig. 2 is a block diagram of the digital camera to PC via bus coupling of the invention; 
Fig. 3a is a horizontal element of a downsampling filter; 
Fig. 3b is a vertical element of a downsampling filter; 

Fig. 4 is a block diagram of an output of a CCD sensor in a primary color layout; 
Pis- 5a 's another embodiment of a horizontal element of a downsampling filter; 
Fig. 5b is another embodiment of a vertical element of a downsampling filter; and 
Fig. 6 is a block diagram of an output of a CCD sensor in a secondary color layout. 

DETAILED DFSCRTPTION OF TH E PRFFF.RRFD EMRODIMFMTS; 

Referring to Fig. 2, an embodiment of the invention is shown. As is evident from Fig. 2, 
digital video camera 20 does not include any DSP hardware. Any necessary transformation of 
raw digitized CCD video data to NTSC (YIQ) or PAL (YUV) format preferably is done using 
appropriate software running on PC 28. In an embodiment PC 28 further performs additional 
manipulations of the video data as discussed further below. 

^•'^eo camera 20 has lens 21 for focusing on a scene or the like denoted by optical path 
22. CCD sensor 24 generates an analog video signal which is passed to correlated double 
sampling and automatic gain control device 23 for conditioning. The conditioned raw CCD 
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analog video signal is converted into a raw CCD digital video signal by A/D converter 26. 
Camera 20 also preferably includes microcontroller 25. 

Color filters are preferably placed over CCD 24 to generate color analog video signals. 
As noted above, the color filters may be in primary or secondary color layouts. When using a 
5 primary color layout, the sensor output typically produces alternating lines of red, green, red, 
green, red, green . . . (R,G,R,G,R,G...) and green, blue, green, blue, green, blue 
(G,B,G3,G,B.,.) colors represented as analog color video signals. The raw analog color signals 
from CCD 24 are converted into raw digital signals by analog-to-digital (A/D) converter 26. 

In one complimentary color layout, CCD 23 typically produces alternating lines of G+Cy, 
10 Mg+Ye, G-fCy, Mg+Ye, G+Cy, Mg+Ye ... and Mg+Cy, G+Ye, Mg+Cy, G+Ye, Mg+Cy, G+Ye 
. . . color video signals. The analog color video signals are digitized by A/D converter 26. The 
raw digital CCD color video signals in a secondary color layout are further processed such as to 
convert them into the RGB color space. In one embodiment, camera 20 includes the necessary 
processing hardware and/or software required for converting the secondary colors into an RGB 
1 5 format of video data. In another embodiment, PC 28 converts the raw digital CCD video data in 
complimentary color layout to RGB data, for example under software control. 

For example, in the latter embodiment, digital camera 20 is coupled through a bus having 
limited bandwidth, for example USB bus 27 to PC 28, Digitized raw CCD sensor video data 
from A./D converter 26 is transmitted over bus 27 for processing prior to display or storage. For 
20 example, PC 28 preferably generates raw luminance and color difference signals from the raw 
digital CCD signals in the complimentary color layout. PC 28 converts the color difference 
signals to RGB values which then can be transformed into a composite video format, such as 
NTSC (YIQ) or PAL (YUV). Additional processing may be performed by PC 28 on, for 
example, the RGB color signals, such as white balancing and gamma correcting. The video data 
25 may be displayed, for example on a video monitor associated with PC 28 (not shown.in Fig. 2). 
For a funher disclosure of the conversion of raw digital CCD color signals in a complimentary 
color layout to RGB signals, and processing of RGB color signals, see commonly owned U.S. 
Application No. 08/520,61 1 , filed on August 30, 1995, incorporated herein by reference. 

The raw digitized CCD video data transmitted over bus 27, whether iji a primary or 
30 secondary color layout, has 8 bits of color information per pixel. The 8 bit size of each raw 

digitized CCD color signal is of course less than video data transmitted in, for example, YUV9, 
YUV 1 2 or YUV 16 formats, which effectively have 9, 12 and 16 bits per pixel, respectively. 
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However, even though the raw digitized CCD video information has fewer bits per pixel than 
any of the YUV formats, it has more color information in those 8 bits than any of the YUV 
formats, which have a single pair of 8 bit color difference numbers, U and V. shared by 1 6, 4 and 
2 pixels, respectively. It can be seen, therefore, that the conversion of the RGB color signal into 
5 a YUV or YIQ formal by DSP 16 in conventional systems as shown in Fig. I inefficiently 

generates redundant information which nevertheless must be transmitted over bus 19, thereby 
occupying valuable bus bandwidth. 

In a preferred embodiment shown in Figs. 3a and 3b, the digital video signals which 
represent raw CCD digital video signals are subjected to low-pass or downsampling filtering by 
10 filter 30 to decrease the resolution of the video image being transmitted over bus 27. 

Downsampling as herein described fiirther decreases the bit-rate and'enhances an ability to use a 
limited bandwidth bus 27 for transmission of video data to, for example, a PC at a sufficiently 
high refresh rate. 

The horizontal position 3 1 of a combination downsampling filter 30 is shown in Fig. 3a. 
1 5 A/D converter 26 of camera 20 having primary color layout produces alternating lines of 
RGRGRG and GBGBGB digital video data, representing raw sensor output as shown, for 
example, in Fig. 4. The digital video data is horizontally downsampled by horizontal 
do\^'nsaimpling filter 3 1 , which decreases by approximately 33% the horizontal resolution of the 
video image. For example pixel element 0,0 is multiplied by 0.5 (or divided by 2) by multiplier 
20 32. 1 and the result is passed to adder 33.1. Similarly, pixel element 0,2 also is multiplied by 0.5 
(or divided by 2) by multiplier 32.3 and the resultant value is transmitted to adder 33.1 . 
Multipliers 32.1 and 32.3 may be hardware devices, for example a shifter, for shifting right the 
digital value of pixel elements 0,0 and 0,2. In another embodiment, the multiplication or 
division may be implemented in software and under processor control. Pixel element 0,1 , which 
25 is a green pixel element is multiplied by multiplier 32.1 having a coefficient of 0 so that the red 
pixel elements are separately downsampled. 

Adder 33.1 combines the values from multipliers 32.1, 32.2 and 32.3 to provide a single 
digital value 34 representing downsampled pixel elements 0,0 and 0,2. 

Similar downsampling filtering is performed on groups of pixel elements 0,2 - 0,4; 0,4 - 
0,6; 0,6 - 0,8 . . . for the entire first horizontal row 0 as shown in Fig. 3a. The entire filter matrix 
is then shifted a single pixel element to the right and the green pixel elements of the first 
horizontal row are downsampled. For example, pixel element 0,1 is multiplied by 0.5, pixel 



30 
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element 0,2 is multiplied by 0 and pixel element 0,3 is multiplied by 0.5. The results of the 
multiplications (or divisions as described above) are added to provide, for example., a single 
digital value representing downsampled pixel elements 0,1 and 0,3. Downsampling filtering is 
correspondingly performed on pixel element groups 0,3 - 0,5; 0,5 - 0,7; 0.7 - 0,9... for the 
entirety of row 1 . 

After downsampling horizontal row 0, rows 1 - n are downsampled in an identical 
fashion. 

After performing horizontal downsampling, a preferred embodiment of the invention 
vertically downsamples using a vertical downsampling filter, such as vertical downsampling 
filter 39 of downsampling filter 30, for example to decrease by approximately 33% the vertical 
resolution of the image to be transmitted over bus 27. An embodiment of a vertical 
downsampling filter 39 is shown in Fig. 3b. Pixel elements 0,0, 1 ,0. and 2,0 are multiplied by 
multipliers 35.1, 35.2 and 35.3, respectively, having coefficients of 0.5, 0 and 0.5, and the 
resulting values are added by adder 36.1 to provide a single digital value 37. As discussed 
above, an alternative to multiplying by 0.5 is to divide by 2, a mathematical operation easily 
accomplished in digital electronics by shifting right. 

The value from adder 36. 1 is the douTisampled value of pixel elements 0,0 - 2,0 and niore 
particularly the red pixel elements 0,0 and 2,0. After downsampling all of the red pixel elements 
in column 0, the entire filter matrix is shifted down one pixel element and downsampling is run 
on pixel groups 1,0 - 3,0; 3,0 - 5,0; 5,0 - 7,0; 7,0 - 9,0, which has the effect of isolating and 
downsampling the green pixel elements in column 0. After down sampling column 0, 
downsampling is performed on columns I - n in the manner described above. 

Downsampling filters of varying configurations are possible, as is evident to a person of 
ordinary skill relying on this disclosure, to decrease the resolution of an image output by digital 
camera 20 by varying percentages. For example. Figs. 5a and 5b show downsampling filter 50 
having horizontal and vertical elements 51 and 59 respectively, for further reducing, as compared 
to the filter of Fig. 3, the resolution of an image in two dimensions. Downsampling filter 
elements 51 and 59 have multipliers with coefficients of 1/3 and 0 for filtering colors 
independently, in the manner described above with reference to Figs 3a and .3b. A decrease in 
resolution typically enables an increase in a frame refresh rate. 

A digital camera 20 having a secondary color layout will generate the color pattern shown 
in Fig. 6. In a preferred embodiment of the invention, downsampling is perf ormed on a 
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complimentary color layout pattern in the same manner as described above with reference to a 
digital video camera having a primary color layout. Any necessary conversion from a secondary 
color layout to, for example, a linear RGB format and an NTSC or PAL format preferably occurs 
in PC 28 after the downsampled, raw, digitized secondary color video signals are transmitted 
5 over bus 27. Conversion to RGB can be done under software control as will be understood by 
those skilled in the art, particularly when referring to Application No. 08/520,61 1, filed on 
August 30, 1995, and incorporated herein by reference. 

The invention takes advantage of the smaller number of bits per pixel of video data 
associated with the digitized, raw output from a CCD of a digital camera as compared to the bits 

10 per pixel of an identical signal which has been converted to a YUV or YIQ format. The 

invention enables transmission of digital video data from a digital camera to a PC through a bus 
having limited bandwidth by moving any YUV or YIQ conversion steps out of the camera and 
into a PC, where they can be done by a high-speed PC microprocessor efficiently and without 
occupying many processor cycles. Advantageously, any other video data manipulations which 

15 are required also can be performed by the PC. For example, software running on an INTEL 

PENTIUM processor operating at 100 MHz was able to do the work needed to convert digitized 
CCD sensor signals into high quality video signals suitable for video conferencing and other 
video processing applications. At typical video conferencing resolution and frame rate, the CPU 
workload was less than 5% of the 100 MHz PENTIUM cycles. By eliminating a need for 

20 sophisticated processing hardware in a digital video camera, the invention provides the additional 
advantage of enabling the manufacture of less expensive digital video cameras. 

It should be understood that various changes in the details, materials, and arrangements 
of the parts which have been described and illustrated in order to explain the nature of this 
invention may be made by those skilled in the art without departing from the principle and 

25 scope of the invention as expressed in the following claims. 
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CLAIMS 

What we claim is: 

1 . A digital video camera, comprising: 

(a) means for generating a plurality of raw analog video signals in response to an image; 

(b) means for converting the raw analog video signals into a plurality oJ'raw digital video 
signals defining the image; and 

Cc) means for transmitting the raw digital video signals to a computer over a bus. 

2. The camera of claim 1 , further comprising: 

(d) means for decreasing the resolution of the image defined by the raw digital video signals 

3. The camera of claim 2, wherein means (a) comprises a charge coupled device. 

4. The camera of claim 3, wherein means (a) further comprises a color filter. 

5. The camera of claim 4, wherein the color filter includes one of the sets of colors of red. 
green and blue, and magenta, yellow, cyan, and green. 



6. The camera of claim 2. wherein means (b) comprises at least one analog-to-digital 
20 converter. 



7. The camera of claim 2, wherein the bus has a bandwidth of not greater than 1 2 Mbits 
per second. 



15 



25 



8. The camera of claim 2, wherein means (d) comprises a downsampling filter. 



9. The camera of claim 8, wherein means (d) further comprises: 

(i) a horizontal downsampling filter to enable a reduction of resolution in a horizontal 
dimension; and 

^0 (ii) a vertical downsampling filter to enable a reduction of resolution in a vertical 

dimension, and wherein each of the horizontal and vertical downsampling filters includes 
coefficients of 0. 
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1 0. The camera of claim 2, wherein the computer has processor means for converting the 
plurahty of raw digital video signals into a plurality of usable digital video signals. 

1 1 . The camera of claim 10, wherein the usable digital video signals are one of YUV and 
5 YIQ signals. 

12. A method for generating digital video signals using a digital video camera, 
comprising the steps of: 

(a) generating a plurality of raw analog video signals in response to an image; 
1 0 (b) converting the raw analog video signals into a plurality of raw digital , video signals 

defining the image; and 

(c) transmitting the raw digital video signals to a computer over a bus. 

13. The method of claim 12, further comprising the step of 

1 5 (d) decreasing the resolution of the image defined by the raw digital video signals. 

14. The method of claim 13, wherein step (a) comprises the step of focusing the image 
onto a charge coupled device. 

20 15. The method of claim 14, wherein the charge coupled device is overlaid with one or 

more color filters, 

16. The rnethod of claim 15 wherein the one or more color filters include one of the sets 
of colors of red, green and blue, and magenta, yellow, cyan, and green. 

25 

1 7. The method of claim 13, wherein step (b) comprises the step of processing with at 
least one analog-to-digital converter. 

1 8. The method of claim 1 3, wherein the bus has a bandwidth of not greater than 1 2 Mbits 
30 per second. 

19- The method of claim 13, wherein step (d) comprises the step of downsample filterin^i. 
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20. The method of claim 19, wherein step (d) further comprises the steps of: 

(i) horizontal downsample filtering to enable a reduction of resolution in a horizontal 
dimension; and 

(ii) vertical downsample filtering to enable a reduction of resolution in a vertical 
dimension, and wherein each of the horizontal and vertical downsample filtering uses 
coefficients of 0. 

2 1 . The method of claim 1 3, wherein the computer comprises a processor for converting 
the plurality of raw digital video signals into a plurality of usable digital video signals, 

22. The method of claim 21, wherein the usable digital video signals are one of YXJV and 
YIQ signals. 



23 A device for processing video signals, comprising; 
' 5 (a) means for receiving over a bus a plurality of raw digital video signals defining an image; 

and 

(b) means for converting the plurality of raw digital video signals into a plurality of usable 
digital video signals, wherein the raw digital video signals are a digitized version of raw charge 
coupled device output signals. 

20 

24. The device of claim 23, wherein the plurality of raw digital video signals have been 
generated by a digital video camera equipped with means for decreasing the resolution of the 
image defined by the raw digital video signals. 

25 25. The device of claim 23, wherein the bus has a bandwidth of not greater than 12 

Mbits/second. 

26. The device of claim 23, wherein the usable video signals are one of YUV and YIQ 
signals. 

30 
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27. The device of claim 23, wherein the digital video camera has an analog to digital 
convener and the raw digital video signals generated by the camera are a raw analog output of 
the charge coupled device which has been processed by the analog to digital converter. 

5 28. The device of claim 23, wherein means (b) comprises a computer processor. 
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FIG. 4 
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